Digitized examinations may be generated by compiling multiple examination items. For example, an item may be a question or problem statement. Items may be compiled from a repository, e.g., a relational database or other electronic storage medium or data store. Items may be categorized by different characteristics, such as field, topic, and/or difficulty of the content. Many times, digitized examinations may be limited or fixed in length, i.e., the number of items presented to a test subject. Nevertheless, in many cases, it is a goal to present a test subject with a broad range of non-cumulative items for the purpose of assessing a sufficient breadth and scope of the test subject's skill and/or knowledge. Accordingly, it is desirable to exclude repeated, cumulative, duplicative, and/or highly similar items in any given digitized examination form. Undesirably similar questions may be referred to as “enemy items.” For example, pairs of enemy items may be linguistically similar, lexically similar, syntactically similar, and/or semantically similar. However, identifying enemy items, when done manually, may be relatively subjective in nature and time consuming, such that resulting digital examination forms may be relatively non-uniform, and difficult to create.
Traditional technology does not provide an automated and/or objective mechanism for identifying enemy items for removal from digitized examination forms. For example, checking for the presence of enemy items on a newly assembled fixed-length test form has been a labor-intensive process. For a complete check, a human content expert would need to compare every item to every other item on the test form. For a 60-item form, this involves
      (                            60                                      2                      )    =      1    ,    770  pairwise item comparisons. The problem of finding all enemy item pairs in a larger pool of items is far more daunting. A pool of 1,000 items contains
      (                                        1            ,            000                                                2                      )    =      499    ,    500  distinct pairs of items, and it's impossible to humanly check every pair for the degree of item similarity between the two. As assessment increasingly evolves from fixed-length forms to computer-adaptive administrations (items being selected from the pool “on the fly” in response to ongoing examinee performance), it becomes not just desirable but essential to find a way to automatically, quickly, and objectively (at the time of administration) determine the degree of similarity between any two given items drawn from an item pool.